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SlJtURY 

II*  consldtr  a  g*a*rali£«d  fora  of  tb*  problaa  of  optloua  linaar 
filtering  and  prediction  for  randoa  proe***e*.  It  i*  ahovn  that,  under 
very  general  condition*,  the  optiaua  linear  eatinatlon  based  on  the 
received  signal,  observed  continuously  for  a  finite  interval  a  1  t  ■  b, 
is  the  lialt  of  optiaus  linear  estiaation  based  on  sasgiled  data  -  l.e. 
based  on  the  received  signal  observed  at  only  a  finite  nu8d>er  of  points 
in  the  interval  a  1  t  ■  b  -  as  the  points  of  observation  becooe  denser 
and  denser  in  the  interval. 

nils  yields  a  aethod  for  obtaining  the  optiaua  linear  estiaation 
in  oases  vhsr*  the  conventional  generalited  tfiener-Hopf  integral  equation 
technique  has  not  been  shown  to  yield  a  solution.  The  relationship  between 
the  snapled-data  solution  and  the  Wiener-Hopf  integral  equation  solution 
(when  the  latter  exists)  is  discussed. 

Also,  a  problsB  is  posed  coocsmlng  the  rate  at  which  the  eiror 
variance  of  optiau*  saag>led-data  estiaates  approaches  the  error  varianee 
of  the  optlaiB  estlaat*  based  on  continuous  obsez^tion,  as  the  saapled 
points  becone  denser  in  the  observation  interval.  This  problea  is  solved 


in  one  case. 


1 .  rVTRODUCTION 
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A.  3tAt— ent  of  the  Problem 

A  l*rg«  clast  of  linear  estiiaatlan  problaais  can  be  cast  in  the 

follovin^  fon:  svq^ae  ooe  is  given  a  random  process  z(t)  in  an  intei^ral 

* 

a  ■  t  ^  b.  A  saj^)le  function  of  this  process  will  be  called  an  obser/ed 
saiq)le.  It  is  assumed  that 

S  i(t)  ■  0  for  a  t  t  i  b  (l) 

i 


0  (s,t)  ■  S  z(8)z{t;  <  06  for  a  ^  8  r  b. 

X  ^ 

a  ■  t  >3  b 


Here  and  in  the  following,  the  notation  l'  denotes  the  ensemble 
expected  value  of  the  quantity  in  bracicets.  Hie  function  0  is  called  the 

X 

covariance  function  of  the  random  profiess  z(t).  We  do  not  assume  that 
z(t)  Is  neceatarily  stationary. 

Also  suppose  there  li  a  random  variable,  q,  with 

K  q  -  0  (.S) 

V  r  - 

B  ^  O  (1.) 

and 


^t)  -  E  qt(t)  <  for  a  i  t  ^  b  (;;) 

llie  problem  is  to  fora  the  optimiaa  linear  estimate  of  q  In  the 

foUcving  sense : 

Mod  an  estimate  q  of  q  such  that 

(a)  'q  is  formed  by  a  linear  operation  on  z(t).  in  the  sense  to  be  defined 


in  Section  I.B;  and 
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(b)  ij^Cq  -  1*  a  mlnloia  for  all  aatlaatas  aatiafylag  (a). 

In  Sactlon  I.B  belov  we  dafine  the  elaee  of  linear  oparationa  on 
z(t)  within  which  the  optlaiB  ie  to  be  found.  Ihie  claae  contalna  all  the 
typae  of  linear  qparationa  which  arc  ueually  considered.  Also,  the 
existence  of  a  unique  which  is  optima  within  this  class  is  guaranteed 
(see  for  exai^le  lef.  1  or  Bef.  6). 

An  exaaple  of  a  nore  fsnl liar-looking  type  of  problea  which  can  be 
represented  in  the  above  form  is  as  follovs: 

Lst  the  randon  process  x(t )  be  called  the  'signal,'  with 

B  x(t)l  .  0  (6) 

w  J 

0^(s,t)  *  1  [^x(s)  x(t )1  oo  (7) 


Let  the  randon  process  n(t)  be  called  the  'noise,'  with 

B|^n(t)  -  0 

0^(e,t)  -  l|^n(s)n(t)|  ^  or 

and 

0^(s,t)  .  l[x(s)n(t)^^  Qo 

Iha  observed  process  s(t )  is 

s(t)  -  x(t)  -f  n(t)  for  a  -  t  k  b 

nms 

0^(s,t)  -  B  Lt(e)  s(t)]  -  0jj(a,t)  ♦  ♦  0jj^(t,s) 


(8) 

(9) 

(10) 


(11) 

(12) 


A  typical  interpolation  or  extrapolation  problea  is  to  estiaate  the 
signal  value  x(tQ)  at  sobs  tins  t^  by  a  linear  operation  on  the  observed 
sample  K(t)  over  the  interval  ^  4he  observation  interval 


P-i20t 
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If  t  Is  outilde  I  a,b 

o  1 


a,b  thla  la  called  Interpolation  (filtering  or  amoothin^j !  problaai; 

,  an  extrapolation  (prediction)  problem. 

Stated  in  the  formulation  of  the  first  few  paragraphs  above,  ve 
would  have  v 


p(t)  =  B 


and  0^(a,t)  given  by  Iq. 


for  which  B 


o 


X  o 


Xil  o 


(12).  We  wish  to  find  a  linear  estljnate  x(t 

pl  ‘ 

x(t  is  a  BLiniaum. 

o  J  t 


(li) 

(1^) 

) 


Firequently  the  time  t^  and  the  observation  Inter'/sd.  'a.ol  are 
regarded  aa  varying,  and  one  is  interested  in  the  manner  in  which  the 
optlMua  linear  operator  on  z(t  )  and  the  Binlmum  error  varlaxice  vary  with 
t^.  a,  or  b  The  formulas  In  the  remainder  of  this  i->aper  will  not 
expllcitl''  reflect  the  variation  with  t^,  since  they  will  be  expressed  In 
terms  of  the  formulation  of  the  first  few  paragraphs.  However,  the 
dependence  of  the  optlsaim  estimator  and  the  mlninum  error  variance  on  t^ 
In  this  case  can  always  be  made  explicit  by  use  of  Bqs  (l2),  (l3),  eund 
(lU).  In  a  similar  manner,  many  other  problems  such  as  estimation  of  the 
derivative  of  the  signal,  etc.,  can  be  translated  Into  the  formulation  of 
the  first  few  paragraphs. 

In  another  problem  which  la  'requently  studied,  it  is  assumed  that 

the  signal  x(t)  has  a  non-rauidoni  component:  2  x(t)j  =  m(t).  where  m(t) 

4a  known  except  for  a  finite  tiumber  of  unknown  but  non-rendcmi  paremeters, 

N 

upon  which  m(t)  depends  linearly  i.e.  m(t)  =  ^  a.P  (t)  where  P  (t) 

k<«  Jv  lu  & 

-1 


are  kncvn  functions.  In  this  case,  the  objective  is  to  find,  say,  the 
optimxmi  linear  estimate  x(t^  )  of  x(t^)  subject  to  the  additional  restriction 
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that  E  identically  in  tiie  unknown  i^arumeter*  .  The 

B»ain  results  below  can  be  extended  to  this  case  in  a  fairly  straightforward 
way.  This  is  left  ae  an  exercise  for  the  interested  reader. 

B .  Deflnltioo  of  Linear  Operation  on  the  Qbser/ed  Process 

Definition.  Given  the  random  process  z{t)  in  the  Inter/al  a  ^  t  ^  b, 
aaaumed  to  have  zero  enaemble  mean  and  finite  covariance  function,  a  (real) 
linear  operation  on  r(t)  we  define  to  be  any  reuidcn  vailable  of  the  form 


1  i  m. 
> 


■ :  =7°'  .(t  <"5 


ini 


where  l.i.m.  atands  for  'limit  in  the  mean,'  c 


(n ) 


■3) 


are  real  conatar  .3,  and 


are  points  of 


a,b|  .  (Here  and  in  the  following,  the  not  ,;an 


a  b  iignifiee  the  closed  inter/al.  a  =  t  ^  b.  ) 

I  J 

All  the  types  of  linear  operations  which  are  usually  considered  are 
special  cases  of  this.  Also,  the  existence  of  a  unique  linear  estimate 
which  is  optiaun  within  this  class  of  estimates  is  guaranteed. 

For  processes  which  are  continuous  in  UiC  aean  over  a  finite  interval, 
an  alteniative  (and  equivalent)  definition  is  as  follows  (Ref.  1);  a 

theorem  of  Karl'unen  states  tiiat  processes  z(t)  wliicli  are  continuoua  in  the 

1 

mean  over  the  finite  interval  i  a,b  can  be  represented  in  U'.e  forr 


z(t ) 


^  ’ 


^^(t ) 

V 


(i' ) 


with  convergence  in  the  mean  f>3r  t  in  a,b  l^^(l)  ^ 

eigenfmnetions  and  eigenvalues  of  the  integral  equation 

V 


0  (  F 


f  c  (.St)  i^(t)fit 


(IT) 


and 


.'ire  randcm  variables  glvaiby 


V 
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vlth 


t 


r(t )  0^(t  )dt 


(18) 


xi'z  I  ■  0;  8  z  z  1 

i  V  j  .  . 

A  liiM&r  operation  on  the  process 
rariable  of  the  fom 


-  6  (Xroneckor  delta)  (19) 

z(t)  is  then  defined  to  be  a  random 


where 


,  v 


V>«1 


o( 


z 

V  V 


(20) 


l^re  are  useful  applicatlona  for  both  of  these  definitions  of  linear 
operation  to  the  linear  interpolation  and  extrapolation  problem.  I^e 
opt  inns  linear  estiisate  q  and  the  minianim  error  variance  |j  can  be  found, 
for  processes  which  ore  continuous  in  the  mean,  by  utilizing  the  represen¬ 
tation  (l6);  (see  Ref.  1,  section  6  and  the  Appendix  to  the  present  paper), 
licarever,  for  purposes  of  this  paper  the  flret  definition  above  of  linear 
operation  is  of  main  interest. 

C .  The  variational  or  'Integral  Squation’  Method  of  Solution 

What  Blight  be  called  the  conventional  method  of  solution  of  the 
problem  posed  in  Section  l  A  is  as  foLlows:  one  considers  linear  operators 
of  the  fora 

q  X  ^  z{r)  k(t)dT  (21) 

Iraa  in  the  simplest  cases,  when  dealing  with  finite  observation  intervals, 
it  is  necesoary  to  assun  that  k  may  contain  delta  functions  of  various 
orders  --  that  is,  that  ^  may  give  finite  weight  to  the  values  of  z(t)  or 
its  darivatires  at  individual  points. 
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/N 


For  eatlaates  q  of  the  form  (21), 


t'Ai  -  q)‘ 


i 


k(r) 


/ 


0  (o,r)k(o)do-  2  p  (r)  dr  4  B  q  (22) 


Denoting  the  estiiikte  vhich  minimlMS  the  expression  In  (22)  by 
nnd  the  corresponding  veighing  function  by  k,  u  applicstion  of  the 

usual  variational  technique  yields  the  following  tntegrel  equation  for  k: 


/ 


t  (z,t)  dr  -  p(t)  (a  s  t  =  b) 

Zi 


(23) 


where  0  and  p  are  given  by  (2)  and  (5).  "Hie  minimum  error  variance 


E 


(q  -  q)  is  given  by 


r  o' 


r  1 

2 


B  (q  -  q)^  -  B  q 

L  i  ^ 


k(t)  p(t)  dr 


(2^) 


Bven  in  the  simplest  cases,  when  the  observation  interval  is  finite 
the  Integral  Eq.  (23)  does  not  have  a  solution  unless  k  is  permitted  to 
contain  delta  Tonctions. 

Zadeh  and  Ragazinni  (Ref.  2)  liave  given  the  solution  to  this  equation 
when  0^  is  a  stationary  kernel  (l.e.  depends  only  on  ,  s  -  t  )  and 
corresponds  to  a  rational  spectral  density  function.  It  turns  out  that 
for  such  cases  1l  .need  only  contain  delta  functions  at  the  endpoints  a  and  b 
of  the  observation  interval.*  This  is  a  fairly  remarkable  fact.  There  are 
important  stationary  random  processes  with  rational  spectral  densities  for 
which  the  sample  functions  are  differentiable  to  a  certain  order,  say  v, 
with  probability  one.  In  such  cases,  one  might  expect  that  the  optimum 
linear  operator  might  be  of  the  form 

\  b 

. '  ,  '  (r)  dK  (r)  (2‘J 

1-0  a 

•  Actually  some  further  conditions  on  0^  and  f  are  needed. 
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( 1 .  th 

vh«r«  ar«  functloa*  of  bounded  variation  and  t  (T)  !•  the  1 — 
derlyative  of  r(T).  The  fact  that  the  optiaun  operator*  in  theee  caeee 
actually  Involre  delta  funct^ona  only  at  the  points  a  and  b  !•  fairly 
reaarkable . 

It  is  of  course  necessaxy  to  Justify  the  variational  derivation  of 
the  integral  Kq.  (23)  vhen  Ic  is  pensitted  to  contedn  delta  functions. 

'niat  is,  even  if  (23)  hsis  a  solution,  it  is  in  such  cases  necessary  to 
give  a  separate  proof  that  the  satisfaction  of  (23)  is  in  fact  a  necessary 
aad  sufficient  condition  for  the  elniaitation  of  the  expression  in  (22); 
see  for  example  Ref.  4.  Also,  one  should  prove  that  the  optioxun  estimate 
derived  In  this  vay  is  actually  the  optisoim  of  all  possible  linear  estlnates 
and  not  Just  the  optlnum  sjsong  estimates  of  the  form  (2l).  This  can  be 
proved  in  many  cases. 

For  general  kernels  0  (■  t),  however,  one  cannot  assume  that  (23) 

z 

has  a  solution  even  if  ^  is  permitted  to  contain  delta  functions;  nor  can 
one  assuee  that  the  optimum  linear  estimate  is  of  the  form  {2'-)),  say. 

It  would  be  very  desirable  to  exhibit  a  method  for  obtaining  the 
optlmiB  linear  estimate  4,  without  making  any  special  etssumptions  as  to  the 
fora  of  0^.  The  main  purpose  of  this  paper  is  to  exhibit  such  a  method, 
subject  only  to  the  veiry  general  restriction  that  the  process  z(t)  be 

r  t 

continuous  in  the  mean  over  a,b  (The  Appendix  briefly  discusaes  an 

SLltemative  method  for  obtaining  4'  ) 


V 
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1 1 .  OPTIMUM  Lima  ETDUIXS  A3  LJXITS  OF  OPTDCJM  LDOUR  SAMPLID  DATA 
1L3TIMATE3 

8uppo»«  ve  select  n  distinct  points  from  the  intervml 

1  r  ^ 

a,b  .  We  vili  call  this  a  aubdivielon  of  the  intervml  '  a.b  .•  We 

Li  n  .  '  , 

need  not  arnsvaae  that  these  points  are  nissbered  In  order  of  laagnitude. 

An  estijaate  vbich  depends  only  on  the  values  of  f(t)  at  r  finite 
number  of  points  will  be  referred  to  as  a  asuspled-data  estimate.  The  most 
genermd  lineeu:  operation  which  can  be  performed  on  the  set  of  rmiidoa 
variables  i  -  l,...,nj  is  of  the  form 

''  '  °  '  (n )  /  (n ) . 


i2b) 


1-1 


(n  ) 


where  are  real  constants. 


Definition:  We  define  tlie  optlaum  linear  estimate  4^^  of  q,  based  on  the 

observed  values  of  z(t)  at  Just  tbe  points  t  1  -  1 . n,  to  be  that 

random  variable  of  the  form  given  by  (?o)  which  mlnimlies  I  (q^  -  q)*^ 
among  al  1  random  variables  q^  of  the  form  gl.cn  by  (?t).  (See  Ref.  for 
a  detailed  discussion.  ) 

(it  Is  of  course  understood  that  q^  and  4,^  dejjend  on  the  specific 
points  the  number  n  of  points,  but  for  convenience 

this  will  not  be  explicitly  inuicnted  In  the  notation.  ) 

Later  on  ve  will  give  explicit  formolaa  fur  At  this  point,  ve 

wish  to  bring  out  the  fact  that  the  overall  optimum  linear  estimate  q  Is  a 
limit  of  optimum  estimates  based  on  sampled  data. 


•  t  ^  actually  (le;>endo ,  of  course,  on  the  S{)eciflc  points  t 
not  Just  on  tbe  nusiber  of  points. 


(n) 


selected , 


V 
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For  r^f*renc«,  first  give  the  foilovlng  veil  knovn  definition: 

The  process  ^(t)  Is  said  to  be  continuous  in  the  mean  for  a  i  t  '•  b 

'  -  1  '  ^ 

provided  E  ^r(t)  -  2(8)^“^  approaches  lero  as  s  ^  t,  for  each  t  in  a,b  . 

Uveoren  1 .  Let  t(t)  be  continuous  in  the  mean  over  the  finite  interval 


a,b 


Let  If'  i  be  a  sequence  of  subdivisions  of  a,b  .  and  define 

c  n  I  . 


n 


A_  -  len^h  of  maxijBusi  interr^l  between  nei^boriiig 
points  c£  tbs  point  set  consisting  of  a,  b,  t 
. ,  t  ("  )  ^ 


(n)  {2rj) 


(n ) 


where  ^  ^  bhe  points  corresponding  to  the  subdivision 

>t/ Suppose  that  0  as  n  ♦  'nien  the  overall  optimum  linear 

estisate  is  given  by 


Also,  let 


\  -  1.1. ■ 

n 


U 


K  (q  -  q)‘ 


(28) 


Then 


B  -  q)- 


lim  M 


n  ♦ 


(29) 


Proof'  Since  is  a  linear  oi>eration  on  r(t),  ^  is  of  the  form 


n 


4  -  l.i.m.  >  c/“^ 


n  '  ^  i ■! 

( ^ )  '  ( n ) 

where  are  points  of  a,b  ,  possibly  different  fror  t^ 

('rtius,  by  definition  of  'Linear  ojeratlon  on  ^(t),'  ^  Is  the  limit 
in  the  lasan  of  sosg  sequence  of  linear  operations  based  on  sampled  data. 


Ibe  point  of  the  theorem  is  to  exhibit  sequences  of  specific  linear 
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operations  on  saaipled  data  vhlcli  converge  In  aean  to  q.  ) 

By  continuity  In  the  Eaoan  of  r(t),  each  estLawite  of  the  fonu 


n  ( n )  ( n  ) 

,  c^  ;  can  be  ajijjroxliunted  In  th<'  mean  arbitrarily  closely 

'l-l 

by  linear  operations  on  the  variables  for 

sufficiently  large  m.  Thus  .  one  can  without  loss  of  generality  say  that 


q  n  1 . 1 .m. 


'  c,<"*  Zd/"*) 


(.''1) 


i.l 


(q  ) 


for  Bone  set  of  constants  c^  f'or  this  set  of  constants  for  which 

[  'l)  holds ,  let 


’  '  (n)  /.  (n)v 

1=1 


(y^) 


•o  that  q  *  1 . 1 .  m .  (1^ . 
Also  let 


n 


Since  ^  Is  defined  as  the  o}  tlauBi  linear  e8tlp*;to  baoe<i  on  the 


points  of 


,  all  n 


V  ,  ^ 

But,  since  q..  l.l.m.  q^.  It  foliovs  that  I  -  q  -  i.l.m.  ( q^^  -  q 
anj  tnerefort?  llm  i.  .  fliereT'ore  fron.  (  , 


L  lir. 


lirr, 


u 


n 


11 
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Now,  we  wish  ai*o  to  ehov  that  q  »  1.1. m.  q^.  l^la  will  b«  acconpl lahed 

A  ^ 

If  we  can  ahow  that  I  ^  ‘  0  aa  n  .  .  To  thtb  end,  let 


of-  -  (1  -  k) ^ 


(3o) 


where  k  is  any  real  nunber. 
Then , 

o(  •  -  q 


(1  -  k  )  (4^  -  q)  f  k(<l^  -  q) 


(37) 


Thei^fore , 


r 


«,(*<•-  q)‘ 


where 


and 


u 


Q 


'T  V 

U  U 
n  n 


(%  • 


C  *3 


(36) 


(3)) 


(40  ) 


Then 


*  ,(  «•  -  q)‘ 


1  ‘  2k 


.  -  ?■  (ui) 


Iherefore,  It  auat  be  true  that 


F 


c*  ,  C  « 


(42) 


i 

since  otherwise  it  would  be  posstble  to  choose  k  bo  that  I  -  q)  j<;i  . 

which  is  ijiposslble,  since  £?<  '  is  e  linear  estinate  baaed  Just  on  the 

points  of  a/  . 

n 


ir 


By  (Up  )  emd  (5?  '  , 


E  (q^  -  q )  (4^  -  q )  -  m 


(U/. 


Therefore  , 


'  \  -  s. 


\2i 


Therefor*,  by  (i^),  K  (q^ 
Theorem  i . 


M  -  U 
n  n 


« 0  as  D  ' 


(UU  ) 


■nils  proves 


Corollary  1 :  Let  C  by  any  class  of  linear  estlnates  of  q  which  contains 
all  cittmtes  of  the  form 


h 

q  .  t(r)  dK(  r) 

'a 

where  K  Is  of  bounded  variation.  If  there  exists  in  C  an  estlaate,  say 
which  is  optlaum  among  all  eBtlaates  belonging  to  C.  tdieu  q  Is  the  overall 
optimum  lineai-  estimate  q 

Proof :  This  is  an  Lnaiediate  consequence  of 
(a)  the  uniqueness  of  q, 

(b  )  the  fact  that  all  sampLed-data  estlmstei!  belong  to  any  such 
class  and 
(c  )  Theorem  1 . 

Specific  fomulaa  lor  the  optimum  aamj) led -data  estimates  q^  are  as 
foilowf  (see  also  Ref  .' )  ■ 

'onslder  the  subdivision  of  a.b  /I th  uo  1  nts  t ,  ^ ^ . t  \ 

n  I  n 


( n  )  ( n  ) 

\  .  1/ 


(n) 

(n  ) 


Define 
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It  is  dssired  to  find  constantB  c 


.  (n) 


such  thAt  K 


1-1 


is  mlBimlMd.  In  the  following,  in  order  to  avoid  c  ircim’ ocut  1  one  ,  it  will 


be  Ass\aMd  that  the  astrlx  ^  nonslngulAr.  Of  course,  this 

assvflqrtioo  is  not  tvecesssj-y  for  the  existence  of  a  unique  optimun 
(although  if  (0j^j  )  aingulsu-,  the  set  of  constants  giving  q^  will 

not  be  unique).  Hc^rever,  assuning  nonsingularity  of  (0  )  for  convenience 


the  process  of  finding  d 
result  being 


(n) 


1 


iij 

is  a  simple  differentiation  problem,  the 


Theorem  2:  Let  (0  ,  .  )  be  nonalngular,  and  let 
-  lij 


Then 


ti ' 


(0  ) 


-1 


(matrix  inverse  ) 


(•'M 


117  "l 


(-■) 


r 

■  % 


'  z^ij  ^,1 


1 ,  J  “i- 


(^<3) 


It  is  Interesting  to  note  what  happens  hear',  st  i  caily  when  n  la 
allowed  to  approach  infinity  (and  defined  by  (2',  )  approaches  zero)  in 
(  Uv  )  and  (U6).  We  have 


n 


where 


d  -  h,  t,  and  u  »  K  q" 

^  ,  1  i  Q  I 

1-1 


’  ,  'rij  Pj 

J 


^  Pi 


i  "  1 


'rtiat  is,  )  is  the  solution  of  the  equation 


0  k 


la  tha  Halt,  tbmu  (of  coutm,  on  a  purely  bauristlc  basis), 
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Ihls  is  saan  to  ba  tha  rasult  axprassad  ia  (23)  aad  (PU)  dartaad  by 
tba  'convantioaal'  Tariational  taehaiquas.  Of  coursa,  this  passaca  to  tha 
Halt  is  UDjustlfisd  if  tha  rasulti^  latagral  aquatioa  doas  aot  hava  a 
solutioo.  If  tha  iatagral  aquation  has  a  solutioo,  aad  if  it  can  ba 
prowad  that  this  solutioo  actually  solves  tha  problaa  of  ainiaisiag  tha 
axprassiao  la  (22),  than  this  passa^ta  to  tha  lialt  Is  actually  Justified. 
For  a  dlseusslao  of  tha  'Intacral  av*atiao'  aathod  of  solution,  saa  for 
aza^pla  Saf.  k. 

An  Intarastiac  problaa  which  now  arises  is  tha  foUoaing:  given 

.  r  2'  f  1 

p,  I  |q  ,  and  a  subdivision  t  of  >  establish  an  uppar  bound 

for  -  ii*  If  a  sr^plad-data  astiaata  4^  is  used  as  an  appraalaatian  of 
It  would  ba  iatarsstlng  to  know  how  far  short  this  falls  of  tha  pptlaua 
astiaata  q.  \  It  would  ba  daslrobla  to  obtain  such  a  bound  directly  In  tans 
of  tha  properties  of  0  and  r,  without  having  to  know  explicit  axprasalons 

K 

la  cloned  fon  for  q  and  u.  this  problaa  aaaas  difficult  to  treat  In 
gaaaral.  An  axaapla  of  this  type  of  rasult  is  tha  following: 


(.,t) 


3:  Ut 
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••1  l*t  p(t)  b*  at  laaat  thraa  tlaaa  dlffarantlabla  and  satisfy,  for  t  in 
tbs  flnlta  intarval  > 


I  p2(t  ♦  h)  -  p^(t)| 
p(t  4  h)  ^t  ♦  h)  -  p(t)  p(t)  j 
p(t  ♦  h)  p(t  +  h)  -  p(t)  ‘fS(t)  j 


(49) 


Him 


♦  p(a)  ’i^a)  4 


"1 

p(b)  ?(fe)j 


(50) 


It  is  aasily  saan  that  this  is  a  rmtbar  analagant  rasult.  An  outline 

of  the  proof  is  as  follows:  For  the  particular  case  assuaad  in  the 

the  or— ,  it  is  possible  to  explicitly  invert  the  aatrix  (0  ,,)  **  ^  5/ 

and  thus  cat  an  explicit  axpressloo  for 

n 

?.ij  Cl  pj 

lJ-1 

Vs  th—  evaluate  il  -  lia  U  directly  as  la  Bef .  5,  except  retaining  the 

n 

b  *  ft 

first  order  ter—  In  5  -  — - —  .  Bie  result  cc— s  out  as  stated  in  the 

n 

thsor— .  Unfortunately  this  asthod  of  proof  Is  of  very  special  applicability, 
In  that  In  ge— ral  a—  —not  expect  to  be  able  to  Invert  h  U.0  . 

k—wlndgs  of  1!^  in  oloaed  font  is  Involved  la  the  proof. 
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If  tha  proccH  t(t)  aod  the  nadca  variable  q  are  Jointly  Qauasian, 
than  the  optiBviB  llnaar  aatlaata  of  q,  la  actually  optUna  aaoog  a  auch 
vlder  cXaaa  of  aatlaataa.  We  deflna  tha  claaa  F  of  oparatlona  on  t(t )  aa 
f  oUowa : 

Deflnltlop;  F  sontalna  all  raodoB  varlablaa  of  tha  fom 

l.l.a'  ]  (51) 

vhara  t.  ....  t  balong  to  a.b  ,  ,  and  t.  .  ia  a  randoa  varlabla 

■aaaurabla  on  tha  aaaq;>la  apace  of  z(t,  ...,  t(t  and  for  which 

r  -  In 

'  2 
“  [^*‘(n) 

Thaoraa  <»;  Let  tha  proceaa  z(t)  and  tha  raadcB  variable  q  ba  Jointly 
Oaunalan.  Alao,  aa  before,  let  ba  the  optlaua  linear  aatlaate  of  q  and 
let 'u  t  I  |(4  - 


.  ,(.-q)^ 


for  all  aatioataa  q  belonging  tc  tha  claaa  F. 
Proof: 


An  aatlaata  q  belonging  to  F  la  of  the  form  (^l).  Let  ua  write  thla 


where  now 


4  °  1.1. 1 

r 


(n), 


(52) 


4  -  . 


However,  according  to  Raf.  6,  pp.  if  z(t)  and  q  ara  Oauaaian, 


L(^-q)"J  ^  1 


where 


.... 


(51*) 

(n), 


q^  ia  the  optiaia  linear  aatlaata  baaed  on  Just  tha  variables  z(t^'°'), 
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■  2  ^  2 

Now,  slBce  q  -  i.l.m.  th«n  E  (q  -  q)  j  “  11a  E  (q^  -  q)  ,  • 

-  q)^j  .  Also 

U  ^  E  Cq  “  q)  Itt^  E  C4  -  q)  This  proves  the  theorem. 

i  J  I  “  I 

W>  cam  also  state 

Theorwi  Let  the  process  z(t )  and  the  ranf.oni  variable  q  be  Jointly 
Oauaslan.  Let  be  a  sequence  of  subdivisions  of  the  finite  interval 

a,b  aatlafyln^  the  conditions  of  Tlieoram  1  and,  In  addition,  having  the 

l  i 

property  that  v'  is  a  reflneiaant  of  v'  for  m  ^n.  Also,  let  0  be  the 
ana 

optlauB  linear  estimate  of  q  baaed  on  the  points  of  v'  'Rien 

q  -  lljn  q^ 

with  probability  one. 

Proof:  This  follovs  directly  from  a  theorem  of  Doob  (Ref.  ,  pp.  232-233). 


But 


from  (^).  Inf  E  Cq^^  -  q)‘ 


IIJD  E 


s 
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APPPPDC 

The  Karhunen  rei"^»entatloo  of  processes  vhich  are  continuous  In  the 
mean  can  be  used  to  obtain  q  as  follofWB; 

We  Lave,  for  z(t  )  continuous  In  the  mean  over  a,b  ,  (a,  b  both 
finite  ), 

0  (t) 

z{t)  z,  — ^ -  .  te  a.b  (kl) 

.>v  ■  •' 

with  convergence  In  the  mean.  The  0^  and  ■  are  eigenfunctions  and 

eigenvalues  of  the  Integral  equation 

b 

0(t )  ‘  0  (s,t  )  0(b  )  ds  (a?) 

z 

a 

The  random  variables  z  are  given  by 

'  b 

z  =  zU)  0  (t;  dt  (A5^ 

V  ■  i  V 

a 

Also 

S(zJ  -  0  (A4) 

K(z  z  )  3  6 

M  '  U  V 

We  now  let 

\ 

4  -  z  with  ^  ‘  -  (A‘)  ) 

.VI  .1 

V-I  1 

and  deteralne  to  thac  E  (q  -  q)*^  is  minimized.  The  resulting 

constants  will  be  denoted  by 
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Ibus, 


r  1  r  V-  1 

I  j(4  .  “  I  W  '\*v  ■ 

-  Va^  ^  ♦  1  q® !  -  sV’i  »  q* 

^  V  '‘I  L  ''  '' 


■  Iq 

L 


I  qs 

V  V 


a;  r 

A.  -  I  N 


Alao, 


•o  tbat 


ntarafor* 


-  I  [qr(t)]  1  qxj  0^(t) 


r 

I  '  qt 


p(t)  0  (t)  dt 


-  ^  \  z  (oonvargaoce  In  mmmn) 


■  -  f 


p(t)  0  (t)  dt 


and  z  glran  by  (AS).  Also, 


I  '  \  , 

2  \  j  2 


*  1  Q  )^ .  »  u  •  I  1  -  /  oc 

I  j  I  i  /  -i 


(AlO) 
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liIuAtians  (AS)  aad  (A9)  eoxild  be  vritten 

b 

li  -  i(t)k^(t)dt  (All) 

'a 

where 

D 

1 

This  la  very  alallar  to  the  result  that  would  be  obtadjiad  if  ooe 
tried  to  ' solve '  the  integral  Iq.  (23)  by  the  aetbod  of  aigeofunotloo 
expansions.  The  resultias  'solution'  for  k(t)  would  be 

Ut)  (A13) 

1 

vlth  gives  by  (A9).  The  trouble  with  this  is  of  course  that  the  sun 
on  the  right  side  of  (AI3)  usually  does  not  converge  in  any  ordinary  sense 
to  a  weighting  function  which  corresponds  to  "q.  In  other  words,  although 
4  is  the  limit  of  operations  on  B(t)  which  correspond  to  the  weighting 
functions  k^,  the  limiting  operation  4  does  not  correspond  to  any  weighting 

-  j 

function  k  which  is  the  limit  of  the  a  in  the  ordinary  sense. 

n 
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